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ABSTRACT 

In order to provide teachers with support for the 
improvement of computer-based problem solving, this paper considers 
the following research questions — and provides answers — about how to 
teach students problem solving and how to design training courses for 
teachers: (1) What are the teachers' and researchers' assumptions 
about the relation between teaching and learning?; (2) What 
constitutes "good teaching" in technology-based classrooms where 
problem solving is a goal and what do "good" teachers do that their 
colleagues don't do?; (3) What is the relation, if any, between "good 
teaching" and students' ability to problem solve successfully?; (4) 
Have schools or school systems successfully overcome teachers' 
resistance to using technology and how successfully have those 
teachers been trained?; (5) What results, in terms of student 
learning, have accrued from change strategies directed at increasing 
or improving the teaching of problem solving?; (6) What types of 
training for what duration and with what frequency with what types of 
teachers — veteran teachers, new teachers, teachers who believe in 
"hands-on" and teachers who "go by the book" — produce "good 
teaching?"; (7) How do different training strategies produce positive 
changes in students' problem-solving behavior?; (8) What different 
types of training strategies need to be employed to deal with 
differences in teachers' beliefs about what should be taught and how 
it should be taught?; (9) What types of tra ining are needed to 
provide teachers with both the mathematical knowledge necessary to 
conduct problem- solving lessons and the pedagogical skill to use 
technology for those lessons?; (10) What differences, if any, are 
there in the outcomes in students' learning to be problem solvers 
when computation is the sole or major focus of computer use versus 
experience with problem solving software, and does ILS use contribute 
to or detract from problem-solving expertise?; and (11) What role 
does the choice of measures play in efforts to learn about the impact 
of technology on students' performance? (Contains 16 references.) 
(Author/AEF) 
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"'Those unable to use ... (technology) face a lifetime of menial work." 
What Work Requires of Schools: A SCANS Report for America 2000, p. 1 5 
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"Despite technologies available in schools, a substantial number of 
teachers report little or no use of computers for instruction. Their use of 
other technologies varies considerably/ Teachers and Technology: Making 
the Connection, p. 1 

"The challenge of integrating technology into schools is much more 
human than it is technological/ Using Technology to Support Educational 
Reform, p. 83 
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Abstract 



Government agencies, corporate executives, educational policy makers, and futurists 
tell us that solving novel problems, using computers as tools for thinking, and 
developing flexibility of thought and action will be key requirements in the next 
century. But available data about students' problem solving show disappointing results. 
In order to provide teachers with support for the improvement of computer-based 
problem solving we need to ask a series of questions — and provide answers — about how 
to teach problem solving to students and how to design training courses for teachers so 
they will incorporate those strategies into their day-to-day teaching. 

Introduction 



For all the early promise of technology, educators seem to be using technology to do 
what they used to do with textbooks rather than reframing education to fit what 
technology can do. The promise of computers as tools for problem solving, in particular, 
seems to have evaporated. I can remember when educators were excited about 
computers. "Recursion wiH allow us to show students what recursion is and let them 
play with it," they said. "Visual hands-on approaches to geometry can happen quickly, 
intuitively, and easily," they declared. "At last we can bring inference, and statistics, and 
probability into the curriculum as we never could before." 

Problem solving, a topic that many professional groups and curriculum development 
specialists had agreed was difficult to teach with textbooks, was viewed as an excellent 
candidate for increased attention in the computer age. Problem solving had long been 
identified as a weak spot in American education. American students did not score well 
on various rounds of the National Assessment of Educational Progress testing program, 
nor did they show themselves to good advantage when compared with students in other 
countries. Problem solving as defined by Polya (1973) and not as defined by the 
standardized tests administered to American school children — which are simply 
measures of how many word problems can be solved correctly — was viewed as an 
important component of the mathematics curriculum, but one which was difficult to 
operationalize within the confines of the traditional textbook-driven curriculum. So 
technology was greeted with acclaim by many educators intent on improving the quality 
and quantity of problem solving in American classrooms. 

Business (1991) has told us that problem solving — the ability to deal effectively with 
novel situations and create flexible, workable, elegant solutions — is a key component in 
maintaining our country's competitiveness. It has been agreed that tomorrow's work 
force needs to know more than which keys to press for which hamburger order, and 
that the Workforce 2000 and beyond needs to use technology as part of a general 
problem-solving process. So more than a decade after the introduction of computers into 
the schools where do we stand? 
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Surveys of computer use have shown us that problem solving via computer hasn't 
been widely accomplished (Anderson, 1993). To be sure, there have been exciting 
developments — many of them chronicled in the OTA report Teachers and Technology: 
Making the Connection (1995). But those developments have been isolated — occurring in 
only a few schools or universities, and not on a systematic basis. According to the OTA 
report, teacher education programs seem slow to incorporate models of good technology 
practice into their classrooms. The OTA report also says today's teachers, in large 
measure, have ignored the technology revolution or have incorporated a few bits of 
technology here and there, but by-passed many significant developments such as 
geometry programs where students can actively explore their questions about 
relationships between lines and angles, or where students can be confronted with novel 
logic problems and work toward the solution of problems that call for more than mere 
computational competency. 

"There is no good software out there," had been a long-time defense against problem 
solving via computers. But professional groups have issued lists of software deemed 
noteworthy. It isn't necessary for teachers to sift through mountains of poor software 
packages to find one or two that may work in their classrooms. 

'There are not enough computers," had also been a long time defense. But recent data 
(OTA, 1995) show that computer-to-student ratios are getting better all the time. Yes, 
some of the computers are 8 bit machines but one can still do worthwhile problem 
solving with 8 bit machines, and a judicious re-allocation of machines around the school 
system will certainly yield computer configurations to address many different types of 
needs (Marshall 1993a). 

The issue, whether we like it or not, is primarily based on human perceptions, 
attitudes, and intentions as Means (1993) pointed out. These issues will not be resolved 
by purchasing additional computers or providing new and different software. Nor will 
they be solved by providing training — ^just any type of training under just any type of 
conditions. We need data, not declarations about the need for teacher training. We need 
models of what works, not mandates for more training. 

Good data tell us what happened, why it happened, the conditions under which it 
happened, and what didn't happen and why. Many teachers are afraid to devote time to 
problem solving because they're not sure about the benefit to their students since 
research on problem solving often suggests that computer-based problem solving has no 
effect on students. So we need to know how good instruction does produce results. 

Many research studies have been compromised by poor designs, inappropriate 
measures, and faulty assumptions. For example. Charters and Jones (1973) offered 
several conditions that educational research must meet: Researchers must ask the right 
questions and they must ask enough questions to describe the situation; researchers 
must make sure that if control groups are used the students in both groups really are 
similar, and that teachers or "the treatment" are described properly when comparison 
studies are conducted; researchers must make sure that "the treatment" was actually 
implemented and implemented in ways described by the project designers. If not, they 
must detail where the differences occurred and what contributed to those differences. 

But research in computer use has failed to follow the wisdom of Charters and Jones's 
suggestions. We are rich in survey data — "How many computers are used and for what 
subject areas?" (Becker, 1983). We have responses and analysis of questionnaire data — 
"How has using computers changed the way you teach?" (Hadley and Sheingold, 1993). 
We have learned about innovative ways to deliver training, but we have no idea how 
different strategies for training can and do produce intended results. 

Before we begin to design and deliver large-scale training (if indeed, the money ever 
becomes available), we need to answer some fundamental questions. Since computer- 
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based problem solving was viewed as an important educational step forward, but seems 
to have been avoided by many technology-using teachers, asking questions about 
problem solving is a good place to start a rigorous examination of what's happening and 
why in technology-equipped classrooms. 

While work on a research agenda has been initiated within the International Society 
for Technology in Education (ISTE) (Roblyer, Smith and Robin, 1995), the work fails to 
acknowledge such key issues as the epistemological assumptions of the researchers or 
teachers and ignores many important issues at the classroom level, so this paper seeks to 
continue and amplify the discussion about research, its aims and operations. 

Research Questions About Problem Solving 

Research Question #1 

What are the teachers' and researchers' assumptions about the relation between 
teaching and learning? 

For a long time, the behaviorist paradigm has dominated education in general and 
educational research in particular. We must acknowledge that good problem-solving 
instruction won't fit the behaviorist mode of teaching and many traditional types of pre- 
/post-studies are not robust enough to capture the effects of good problem-solving 
instruction; nor are the measures used sufficient enough to capture the effects of good 
practice (Marshall, 1993b). Research studies are needed that fully describe the different 
types of problem-solving lessons, the extent to which the teachers followed the goals of 
the software, and the types of thinking strategies students practiced and applied. 

Research Question #2 

What constitutes "good teaching" in technology-based classrooms where problem 
solving is a goal and what do "good" teachers do that their colleagues don't do? 

First of all, let's acknowledge that we have only sketchy criteria for what constitutes 
good teaching in computer settings, especially when it comes to problem solving. 
Responses to surveys that tell us how satisfied teachers are or how dissatisfied they are 
with what they're doing (Hadley and Sheingold, 1993). We also have Guidelines for Good 
Practice (1991), but have we communicated what a "good lesson" looks like? Do teachers 
know what good technology-based problem solving looks like? Is it different from work 
with manipulatives used alone, or with activity books and textbooks? Do we know what 
practices lead to what results in children's thinking as a result of interactions with 
technology? To date, answers to these questions are sketchy at best. 

Research Question #3 

What is the relation, if any, between "good teaching" and students' ability to problem 
solve successfully? 

Models exist for "good" lessons. For example, a "good" week in a math class is 
supposed to consist of the teacher modeling the application of a new algorithm on 
Monday, calling students to the board on Tuesday to work on variations of the 
problems, providing time for seat-work in textbooks and workbooks on Wednesday and 
Thursday, and then testing students on Friday. Is that a good model for problem solving? 
If not, what do good problem-solving lessons look like? How do we know if those 
models work? Do those models have differential effeas on students? 
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Research Question #4 

How have schools or school systems successfully overcome teachers' resistance to 
using technology and how successfully have those teachers been trained? 

Teachers have developed a resistance to change. Some believe that technology is a fad 
that will go away. Given the revolving door nature of most educational innovations in 
the USA, their beliefs seem reasonable. Teachers have become weary about investing 
their energies in efforts that will disappear when the superintendent leaves for another 
school district. Do we know how to move resistant teachers to becoming computer-using 
teachers? Do we know how to help teachers who use computers for nothing but word 
processing move toward integrating problem solving into their technology routines? 

Research Question #5 

What results, in terms of student learning, have accrued from change strategies 
directed at increasing or improving the teaching of problem solving? 

Training is as trainihg does. For years we have been hearing that training is a key 
element in converting the technology-avoider into the technology-user. But 15 years 
into the technology revolution, we don't have a clear set of guidelines about what types 
of training bring about what types of change in teachers and even less evidence of the 
impact on students of our interventions. We have anecdotal evidence about programs 
that bring teachers into the technology fold, but we have no data on the effectiveness of 
that training on the newly-trained teacher's ability to (1) implement what has been 
presented, (2) implement it in a sustained way and a way that's faithful to the intent of 
the developers of the programs and the goals of the workshop presenters, and (3) 
positively affect student outcomes as a result of that change. 

Research Question #6 

What types of training for what duration and with what frequency with what types of 
teachers — veteran teachers, new teachers, teachers who believe in "hands-on" and 
teachers who "go by the book" — produce "good teaching?" 

Does having a fellow teacher team teach with technology promote better results than 
"pull out" training? Is the cost of training a major variable? Does training before the 
school year begins have more impact than school-year wide training? What types of 
teachers — new-to-teaching or veterans, behaviorists, or constructivists — profit from 
what kinds of teacher training models? 

Do we even know what questions to ask about training? We have little or no data on 
how much of what has been presented in training is actually used in classrooms — and 
what data we have tends to be from small samples or unrepresentative samples. We 
have little or no data on how long newly-trained teachers actually implement the 
changes they've seen. We do have evidence (Marshall, 1996) that some trainers fail to 
understand the software packages they themselves are using in training situations. 

We need more information on the specific features of training programs that produce 
reproducible results. Good faith and good intentions are insufficient when we consider 
the demands for good practice. We need systematic evidence of what is necessary and 
what is sufficient in training situations. 

Research Question #7 

How do different training strategies produce positive changes in students' problem- 
solving behavior? 
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What "we, the trainers" know is not what "you, the teachers" value. Let's be clear at 
the outset. Data show us that mathematics is taught for less than 20 minutes a day in 
some classrooms and that computation lessons — primarily "seatwork" — dominate the 
instructional plan. Lessons devoted to skills recommended by professional organizations 
(1989) — geometry, probability and statistics, logic — are seldom, if ever taught. Textbooks 
still tend to treat these topics, if at all, by including chapters at the back of the book — a 
clear signal that teachers can take it or leave it. Now maybe this pattern of 
computational dominance is rooted in the fact that teachers haven't been adequately 
trained to do more than computation. But we need to know if these issues are 
implicated in our students' failure to problem solve successfully. 

Teachers teach what they believe in or what their communities value. Computation is 
still highly valued as a skill, and many teachers and parents believe that computation — 
the "one right way and one right answer" approach — is the only appropriate subject 
matter, especially at the elementary school level. Take 20 minutes per day for 
instruction, do a little computation, and there's not much time left over for other 
mathematical activities. Do these practices compromise students' ability to problem 
solve? If so, how do we encourage teachers to teach beyond computation and, in the 
process, enhance students' problem-solving abilities? 

Research Question #8 

What different types of training strategies need to be employed to deal with differences 
in teachers' beliefs about what should be taught and how it should be taught? 

Some teachers are firm believers in a behaviorist approach to teaching and learning. 
For many of them, problem solving instruction, at least in the elementary schools, is not 
an appropriate use of students' time. Other teachers say teaching problem solving will 
have little effect on students' performance on standardized tests — the benchmark against 
which many instructional activities are weighed. How do we promote an acceptance of 
problem solving in those Classrooms? And does that acceptance translate into improved 
problem solving ability for our students? 

Research Question #9 

What types of training are needed to provide teachers with both the mathematical 
knowledge necessary to conduct problem-solving lessons and the pedagogical skill to use 
technology for those lessons? 

Once there was a "cottage industry" approach where software developers who thought 
they had important ideas that had not been addressed in textbooks developed interesting 
ways to present those ideas via technology. But we often found that the topics 
introduced were new to many teachers and frightening to some because their own 
educational experiences in mathematics had not been rich enough to give them a 
foundation in a broad range of mathematical topics. 

Today we have the increasing presence of large-scale producers of Integrated Learning 
Systems (ILSs) that tend to present computationally-based activities to the exclusion of 
problem-solving activities. At this time many believe that the ILS producers will 
eventually dominate the market to a greater extent than they do today. Some would say 
that to invest time and energy in learning problem solving and learning to teach 
problem solving, is a waste of time because in practice schools will ignore the demands 
of business leaders and government officials and follow the lead of the ILS producers. 
How do we help teachers acquire expertise in seleaing topics and delivering lessons 
based on good problem solving of their own design instead of relying on ILSs? 
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Research Question #10 

What differences, if any, are there in the outcomes in students' learning to be problem 
solvers when computation is the sole or major focus of computer use versus experience 
with problem solving software, and does ILS use contribute to or detract from problem- 
solving expertise? 

We have no idea how much exposure to computers and in what pattern — ten minutes 
every day, half an hour every few days, a hour a week, none of the above — produces 
consistent lasting effects of instruction. We don't know if computer use needs to be 
buttressed with complementary work with manipulatives — and, if so, at what grade 
levels that use is needed — or supplemented by films, books, field trips, etc. We need to 
design studies that explore how students learn problem solving via computer. 

Research Question #11 

What role does the choice of measures play in our efforts to learn about the impact of 
technology on students' performance? 

"Why should I teach it when they don't test it?" This question has challenged 
innovators designing problem-solving curricula for over 30 years. Many curriculum 
development projects that received massive infusions of federal dollars were stymied 
when school systems demanded hard data to show that students taught problem solving 
were performing better than their peers who were doing their multiplication tables 
every day (Marshall, 1994). The problem faced by the curriculum developers had several 
facets: schools tend to trust the standardized tests currently used by most school systems 
in the US and only those standardized tests; those standardized tests have a sub-test 
called "problem solving" but the sub-test merely tests students' ability to solve word 
problems. The tests are not sensitive enough to detect differences in overall 
mathematical ability acquired from problem-solving curricula. As a result, many of the 
projects were deemed to have failed because students' scores in "problem solving" did 
not rise sufficiently to justify the school systems' continuance of a problem-solving 
approach. 

The fact that other measures were and are available — all standardized, all subjected to 
appropriate methodological practices — did not deter schools and school systems from 
relying only on "traditional" tests. The fact that other measures could be statistically 
equated with traditional tests so we could have indices of both "traditional" test 
performance and problem solving performance has been ignored or avoided by school 
system decision-makers. 

Until we answer these questions and other questions aimed at teasing out the 
multiple, and often competing, factors at work in the design of evaluations, in the 
implementation of technology, in the training of teachers, and the assessment of 
students our efforts at training teachers to use technology's full potential in order to 
equip students for 2000 and beyond will be stymied. 
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